Spatio-Temporal Segmentation with Depth-Inferred Videos of Static Scenes

نویسندگان

  • Hanqing Jiang
  • Guofeng Zhang
  • Hujun Bao
چکیده

Extracting spatio-temporally consistent segments from a video sequence is a challenging problem due to the complexity of color, motion and occlusions. Most existing spatio-temporal segmentation approaches rely on pairwise motion estimation, which have inherent difficulties in handling large displacement with significant occlusions. This paper presents a novel spatio-temporal segmentation method for depth-inferred videos. The depth data of input videos can be estimated beforehand using a multiview stereo technique. Our framework consists of two steps. In the first step, in order to make the extracted 2D segments temporally consistent, we introduce a spatial segmentation based on the probabilistic boundary maps, by collecting the boundary statistics in a video. In the second step, the consistent 2D segments in different frames are matched to initialize the volume segments. Then we compute the segment probability for each pixel by projecting it to other frames to collect the statistics, and incorporate it into the spatio-temporal segmentation energy to explicitly enforce temporal coherence constraint. The spatio-temporal segmentation results are iteratively refined in a video, so that a set of spatio-temporally consistent volume segments are finally achieved. The effectiveness of our automatic method is demonstrated using a variety of challenging video examples.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Depth Extraction from Videos Using Geometric Context and Occlusion Boundaries

We present an algorithm to estimate depth in dynamic video scenes. We propose to learn and infer depth in videos from appearance, motion, occlusion boundaries, and geometric context of the scene. Using our method, depth can be estimated from unconstrained videos with no requirement of camera pose estimation, and with significant background/foreground motions. We start by decomposing a video int...

متن کامل

Tracking Multiple Self-Occluding People using Dense Spatio-Temporal Motion Segmentation

Abstract. This master thesis discribes a new dense spatio-temporal motion segmentation algorithm with application to tracking of people in crowded environments. The algorithm is based on state-of-the-art motion and image segmentation algorithms. We specifically make use of a mean shift image segmentation algorithm and two graph based motion segmentation algorithms. The resulting motion segmenta...

متن کامل

Motion estimation and segmentation in depth and intensity videos

This paper investigates motion estimation and segmentation of independently moving objects in video sequences that contain depth and intensity information, such as videos captured by a Time of Flight camera. Specifically, we present a motion estimation algorithm which is based on integration of depth and intensity data. The resulting motion information is used to derive long-term point trajecto...

متن کامل

Segmentation-Based Depth Propagation in Videos∗

In this paper we propose a simple yet effective approach to convert existing 2D video content into 3D. We present a new semi-automatic algorithm that propagates sparse user-provided depth information over the whole monocular video sequence. The advantage of our algorithm over previous work comes from the use of spatio-temporal video segmentation techniques. The segmentation preserves depth disc...

متن کامل

EntScene: Nonparametric Bayesian Temporal Segmentation of Videos Aimed at Entity-Driven Scene Detection

In this paper, we study Bayesian techniques for entity discovery and temporal segmentation of videos. Existing temporal video segmentation techniques are based on low-level features, and are usually suitable for discovering short, homogeneous shots rather than diverse scenes, each of which contains several such shots. We define scenes in terms of semantic entities (eg. persons). This is the fir...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2010